Massachusetts Comprehensive Assessment System Research Articles

Relationship Between Cardiorespiratory Fitness, Weight Status, and Academic Performance: Longitudinal Evidence From 1 School District.

We examined the longitudinal relationship between cardiovascular fitness (CRF) and academic performance (AP) among students in a diverse public school district. Furthermore, we determined whether the relationship between CRF and AP varied by student sociodemographic characteristics. This study used data from 2005-2006 to 2008-2009. AP was assessed by Massachusetts Comprehensive Assessment System (MCAS) math (N = 1224) and English Language Arts (ELA) (N = 1248) z-scores. CRF was assessed by an age- and sex-adjusted 20-yard shuttle run (pass/fail). Multilevel growth models were used to estimate the relationship between MCAS z-scores and changes in grade, CRF, and weight status, controlling for student sociodemographic characteristics. Students who passed the fitness test performed better on MCAS tests (ELA: .159, p < .001; Math: 0.346, p < .001) at each study wave relative to those who did not pass. The relationship between AP and fitness was attenuated for low socioeconomic status(ELA: -.111, p < .05) and minority (Math: black -0.249, p < .01; Latino -0.216, p < .01) students. These results add to the literature highlighting the role of CRF as an important predictor of performance on standardized tests. More research is needed to understand the disparities experienced by low-income and minority students.

Teaching to the Student: Charter School Effectiveness in Spite of Perverse Incentives

Recent work has shown that Boston charter schools raise standardized test scores more than their traditional school counterparts. Critics of charter schools argue that charter schools create those achievement gains by focusing exclusively on test preparation, at the expense of deeper learning. In this paper, I test that critique by estimating the impact of charter school attendance on subscales of the Massachusetts Comprehensive Assessment System and examining them for evidence of score inflation. If charter schools are teaching to the test to a greater extent than their counterparts, one would expect to see higher scores on commonly tested standards, higher-stakes subjects, and frequently tested topics. Despite incentives to reallocate effort away from less frequently tested content to highly tested content, and to coach to item type, I find no evidence of this type of test preparation. Boston charter middle schools perform consistently across all standardized test subscales.

The Effects of High-Stakes Accountability Measures on Students With Limited English Proficiency

This article reports on a 3-year ethnographic study in a Boston Public school of the performance of Haitian students on the Massachusetts Comprehensive Assessment System (MCAS) and its lasting and likely impact upon them. MCAS is a mandated exam required for students to graduate from high school. Although there are certain provisions for students with limited English proficiency (LEP) to participate under the mandate, LEP students face the test with an enormous disadvantage compared with other students. The issue of second language acquisition appears to have been completely dismissed in the mandated policy. In light of the challenges faced by students with LEP, the article focuses on the core issues at hand in the Massachusetts standardized exam. It analyzes the complexities of language use and acquisition with respect to students with LEP, the pitfalls of the mandated exam, and the effects and impacts created by MCAS on students and their communities. It suggests that an educational system should take into consideration the complexity of acquiring a second language in a particular social context and develop new testing policies and requirements for students whose first language is not English.

Linking student performance in Massachusetts elementary schools with the "greenness" of school surroundings using remote sensing.

Various studies have reported the physical and mental health benefits from exposure to “green” neighborhoods, such as proximity to neighborhoods with trees and vegetation. However, no studies have explicitly assessed the association between exposure to “green” surroundings and cognitive function in terms of student academic performance. This study investigated the association between the “greenness” of the area surrounding a Massachusetts public elementary school and the academic achievement of the school’s student body based on standardized tests with an ecological setting. Researchers used the composite school-based performance scores generated by the Massachusetts Comprehensive Assessment System (MCAS) to measure the percentage of 3rd-grade students (the first year of standardized testing for 8–9 years-old children in public school), who scored “Above Proficient” (AP) in English and Mathematics tests (Note: Individual student scores are not publically available). The MCAS results are comparable year to year thanks to an equating process. Researchers included test results from 2006 through 2012 in 905 public schools and adjusted for differences between schools in the final analysis according to race, gender, English as a second language (proxy for ethnicity and language facility), parent income, student-teacher ratio, and school attendance. Surrounding greenness of each school was measured using satellite images converted into the Normalized Difference Vegetation Index (NDVI) in March, July and October of each year according to a 250-meter, 500-meter, 1,000-meter, and 2000-meter circular buffer around each school. Spatial Generalized Linear Mixed Models (GLMMs) estimated the impacts of surrounding greenness on school-based performance. Overall the study results supported a relationship between the “greenness” of the school area and the school-wide academic performance. Interestingly, the results showed a consistently positive significant association between the greenness of the school in the Spring (when most Massachusetts students take the MCAS tests) and school-wide performance on both English and Math tests, even after adjustment for socio-economic factors and urban residency.

The Effects of a Computer-Assisted, Schema-Based Instruction Intervention on Word Problem-Solving Skills of Low-Performing Fifth Grade Students

This study examined the effects of computer-assisted, schema-based instruction on the problem-solving skills of fifth grade students with below average problem-solving skills. In a randomized controlled study, 16 students were given a 12-week intervention, and 16 control students received state test preparation review in their regular classroom. Results demonstrated that the computer-assisted, schema-based instruction group showed greater gains relative to the control group on items from the Massachusetts Comprehensive Assessment System and greater rates of growth on biweekly examiner-made probes. However, no difference in gain scores between the groups was observed on the Process and Application subtest from the Group Mathematics Assessment and Diagnostic Evaluation. Implications for intervention and future research are discussed.

Locally-Chosen versus State-Mandated Success Criteria: A Retrospective Evaluation of the Effectiveness of the Success-For-All Program with Fourth and Seventh Grade Hispanic Students

Problem statement: This study compared the growth in reading competencies of Hispanic female and male students in the SFA program as measured by the Massachusetts Comprehensive Assessment System (MCAS) and the Scholastic Reading Inventory (SRI). Approach: The study was a retrospective study with several a priori hypotheses and examined logical comparison groups of 4th and 7th graders, who received 1 and 3 years of SFA reading instruction respectively. Results: This study found that the SFA recommended SRI test presented an over-rosy picture of students reading achievement levels and gains (as compared to the MCAS test) and masked important differential effects of the program. ANOVA results showed that female Hispanic students obtained significantly higher average scores on the MCAS than did male Hispanic students and that these differences between Hispanic female and male students were significantly larger in the seventh grade than in the fourth grade. These findings showed that the SFA program was, most probably, not very successful in developing the reading competencies of Hispanic male students beyond grade 4. Conclusion: The critical and unexamined issue of locally-selected versus state-mandated success criteria for evaluating high stakes programs is also discussed in this article.

Generating Knowledge of Academic Language Among Urban Middle School Students

A quasi-experimental study of a novel, cross-content area vocabulary intervention program called Word Generation showed significantly greater growth among 6th- to 8th-grade students in schools implementing the program than in comparison schools, on a curriculum-specific test. Furthermore, the language-minority students in the treatment, but not the comparison, schools showed greater growth than the English-only students. Improvement on the curriculum-specific test predicted performance on the Massachusetts Comprehensive Assessment System (MCAS) English language arts assessment, again only for students in the treatment schools. Recognizing the need to implement a more rigorous experimental study of this program, nonetheless we conclude that participation in the intervention, with its focus on deep reading, comprehension of current-events topics, productive classroom discussion, developing arguments, and producing persuasive essays, was a plausible contributor to student performance on the MCAS.

Is There a Relationship Between Physical Fitness and Academic Achievement? Positive Results From Public School Children in the Northeastern United States

To determine relationships between physical fitness and academic achievement in diverse, urban public school children. This cross-sectional study used public school data from 2004 to 2005. Academic achievement was assessed as a passing score on Massachusetts Comprehensive Assessment System (MCAS) achievement tests in Mathematics (fourth, sixth, and eighth grade, n = 1103) and in English (fourth and seventh grade, n = 744). Fitness achievement was assessed as the number of physical fitness tests passed during physical education (PE). Multivariate logistic regression analyses were conducted to assess the probability of passing the MCAS tests, controlling for students' weight status (BMI z score), ethnicity, gender, grade, and socioeconomic status (school lunch enrollment). The odds of passing both the MCAS Mathematics test and the MCAS English test increased as the number of fitness tests passed increased (p < .0001 and p < .05, respectively). Results show statistically significant relationships between fitness and academic achievement, though the direction of causation is not known. While more research is required, promoting fitness by increasing opportunities for physical activity during PE, recess, and out of school time may support academic achievement.

Language and the Performance of English-Language Learners in Math Word Problems

In this article, Maria Martiniello reports the findings of a study of the linguistic complexity of math word problems that were found to exhibit differential item functioning for English-language learners (ELLs) and non-ELLs taking the Massachusetts Comprehensive Assessment System (MCAS) fourth-grade math test. It builds on prior research showing that greater linguistic complexity increases the difficulty of Englishlanguage math items for ELLs compared to non-ELLs of equivalent math proficiency. Through textual analyses, Martiniello describes the linguistic features of some of the 2003 MCAS math word problems that posed disproportionate difficulty for ELLs. Martiniello also uses excerpts from children's think-aloud transcripts to illustrate the reading comprehension challenges these features pose to Spanish-speaking ELLs. Through both DIF statistics and the voices of children, the article scrutinizes the appropriateness of inferences about ELLs' math knowledge based on linguistically complex test items.

Connections to the World: Visual Art in Urban Schools

Joy is not a word we often use in conversations about education reform, particularly when we speak of reform in our urban schools. Nor do we often link reform with the synergistic relationship between excellence in academic skills and the abuity to think creatively, critically, or to problefn solve. The fact that the two preceding notions remain mutually exclusive in the minds of some educators has a direct impact on the teaching of visual arts in urban schools. Research has proven some links between academic achievement and the arts and speculated about others (Winner & Hetland, 2000). Linkage is difficult to quantify because we are dealing with the idiosyncratic nature of artistic and cognitive behavior. There is evidence, albeit mostly qualitative, that there is reason to train students to use their minds holistically by having them study subjects that exercise and develop both sides of the brain. The left side, as we know, deals with logic and language. These are the areas of development that are given most attention in school via the work done in math, language arts, and, of course, in preparation for the Massachusetts Comprehensive Assessment System (MCAS). The interpretive, discovery, and exploration modes of learning, addressed in right-brain activities and most frequently used in the arts, helps that development. We know instinctively that our society needs people who can think independently and creatively. We know joy is becoming less and less of a common condition among our school children. And we also know, certainly within the discipline, that students involved in the arts find joy, experience the opportunity to think and interpret, gain the abuity to express and communicate ideas, and often find success not

The legend of the large MCAS gains of 2000-2001.

Issues related to student, teacher, and school accountability have been at the forefront of current educational policy initiatives. Recently, the state of Massachusetts has become a focal point in debate regarding the efficacy of high-stakes accountability models based on an ostensibly large gain at 10th grade. This paper uses an IRT method for evaluating the validity of 10th grade performance gains from 2000 to 2001 on the Massachusetts Comprehensive Assessment System (MCAS) tests in English Language Arts (ELA) and mathematics. We conclude that a moderate gain was obtained in ELA and a small gain in mathematics.

Tips for Readers of Research: Handle Pass Rates with Care

THE ACHIEVEMENT gap receives a lot of attention these days. It is, after all, alleged raison d'etre of No Child Left Behind. It is usually defined as difference between test scores of white students on one hand and those of black and Hispanic students on other. (Asian students almost always get ignored for some reason.) Unfortunately, educators and journalists alike have fallen into trap of reporting changes in pass rates on tests as if they were measures of achievement gap. They are not. At best, passing rates are poor proxies for real measures. The Massachusetts Department of Education; Michael Cohen and Matt Gandal of Achieve, Inc.; David Herszenhorn and Michael Winerip of New York Times; Mayor Michael Bloomberg of New York City; Chancellor Joel Klein of New York City Public Schools; and Achievement Alliance, a consortium of Education Trust, National Council of La Raza, Citizens' Commission on Civil Rights, Just for Kids, Inc., and Business Roundtable, have all reported passing rates as if they were scores. They are not. Consider data from Massachusetts Department of Education, reproduced in Cohen and Gandal's Do Graduation Tests Measure Up? Figure 1 shows passing rates for class of 2003 on Massachusetts Comprehensive Assessment System. On initial administration, when students were 10th-graders, blacks and Hispanics passed at rates far lower than whites: blacks, 37%; Hispanics, 29%; whites, 77%. By sixth retest, passing rates had become much more similar. Cohen and Gandal take these data as showing that the achievement gap closed dramatically. Probably not. Consider following hypothetical data. Hypothetical Data Score Needed to Pass = 60 Pass Average Rate Score Gap Black students, 2004 60% 62 White students, 2004 100% 78 16 Black students, 2005 70% 68 White students, 2005 100% 92 24 Between 2004 and 2005, black students improved both their passing rate (going from 60% to 70%) and their score (from 62 to 78). So far so good. But white students, while not improving their passing rate, improved their score even more, so much that black/white gap in scores (the proper metric) actually increased. There is another problem with conclusions in Massachusetts report. The 10th-graders and 12th-graders are not comparable. Somewhere between freshman and senior years, 18,000 students disappeared from Massachusetts' rolls. Some no doubt moved to other states. But others would have been retained in grade or would have dropped out, perhaps because of failing test. In either case, these low-scoring students are no longer part of class of 2003. Naturally, state officials, who like to brag about students rising to challenge of tests, don't like it when researchers call attention to such dishonest presentation of data, but when Anne Wheelock conducted appropriate analysis of data for Massachusetts for class of 2004, she concluded that state had little reason to brag. She found these passing rates: Class of 2004 Pass Rate All students 74%, not 96% White students 80%, not 98% Black students 59%, not 88% Hispanic students 54%, not 85% Asian students 89%, not 95% Note that gap for final pass rate for class of 2004 is nearly as large as 10th-grade gap was for class of 2003. This is not a trivial case of researchers making statistical mountains out of practical molehills. At least one of journalists cited above knows this. When I presented my hypothetical data to Times' Herszenhorn, he replied, Of course, that's exactly way it works.... We have done some analysis of mean scale scores, enough to know that strong gains in suburbs were masked by looking mostly at who crosses line. …

Comments

Comment by Robert M. Costrell Tom Loveless poses the striking conundrum that just as the social science research is beginning to indicate the promise of test-based accountability, a political backlash is threatening to stop the movement in its tracks or even reverse it. The first part of his paper, on the social science research, provides the promise, and the second part, on political prospects, documents the perils. The paper provides a good overview of where we are and much insightful analysis, if not, in the end, a lot of "hopeful signs." My comments largely draw on my experience over the past five years as an academic in state government in Massachusetts, one of the few states that has successfully instituted high-stakes testing. Since June 2003, students have been required to pass the English and mathematics tests of the Massachusetts Comprehensive Assessment System (MCAS) exam in order to get a diploma. The passing score is low, but the tests are rigorous. Consequently, a nontrivial number of students have been denied diplomas. The vast majority, however, have received diplomas that now mean something. The Cambridge and Brookline boycotts and the high-priced ads of the Massachusetts Teachers Association, of which Loveless writes, did not carry the day. Effects on Achievement Loveless provides a reasonable read of the literature on the effects of standards on achievement. Overall, standards-based reform seems promising, but as it is still early, data are too thin to be definitive. Only a few states have content-based graduation exams (as opposed to the old minimum-skill competency exams). Others have delayed or backed off in one way or another. So it is difficult as yet to gauge the effects of the most rigorous high-stakes testing using [End Page 27] the usual standards of cross-sectional statistical research. If only a few states have high-stakes testing, it is hard to be sure that it is the testing regime, rather than other features of those states, that drives improved performance. Still, in Massachusetts we are encouraged by what is at least a happy coincidence—although we have reason to believe it is more than that—between our high-stakes testing regime and the strong performance of our students, both in levels and improvement, on a variety of external tests, including the National Assessment of Educational Progress and Scholastic Assessment Test.69 We have also generated great improvement on the MCAS test itself, a set of exams that is widely respected (for example, by the evaluations of Achieve) and not easily gamed. These results are consistent with a huge amount of qualitative intelligence on how the MCAS has changed practices on the ground, especially in the urban schools. There has been renewed focus on academic achievement in many concrete ways, at least in English and math, including double-block scheduling, increased writing assignments, greater emphasis on problem solving in math, and improved use of data to identify student weaknesses.70 Based on both the sense on the ground and the data, there is broad agreement among reformers in Massachusetts that although we are still far from achieving the goal of proficiency for all, high-stakes testing has been a key element in raising achievement. This is the view not only of those who had pressed for high stakes all along but also among some who were getting cold feet as the moment of truth approached. There is little doubt among the urban superintendents (some of the strongest proponents of standards-based reform) that the mobilization for such improvement could not have occurred with lesser forms of accountability, such as school report cards. Loveless reports that Margaret Raymond and Eric Hanushek find no statistical difference between states with school report card systems and those with stronger forms of school accountability, but the study did not examine systems of student accountability.71 Unintended Consequences Much of the evidence that Loveless reviews on the effect of standards on dropout rates refers to an earlier generation of standards...

Challenges in the No Child Left behind Act for English-Language Learners

One of the most controversial aspects of NCLB is its performance requirements for subgroups within the general student population. Mr. Abedi and Mr. Dietel examine the implications of these requirements for English-language learners and offer recommendations to help states, districts, and schools facilitate the progress of these students. THE NO CHILD Left Behind (NCLB) Act of 2001 requires that all children, including English-language learners (ELLs), reach high standards by demonstrating proficiency in English language arts and mathematics by 2014. Schools and districts must help ELL students, among other subgroups, make continuous progress toward this goal, as measured by performance on state tests, or risk serious consequences. Through these mandates, NCLB establishes high expectations for all students and seeks to reduce the achievement gap between advantaged and disadvantaged students. These are worthy goals, which require extraordinary improvement in student learning. The challenges for English-language learners are especially difficult, involving both educational and technical issues, including: * Historically low ELL performance and very slow improvement. State tests show that ELL students' academic performance is far below that of other students, oftentimes 20 to 30 percentage points lower, and usually shows little improvement across many years. * Measurement accuracy. CRESST research shows that the language demands of tests negatively influence accurate measurement of ELL performance. For the ELL student, tests measure both achievement and language ability. * Instability of the ELL student subgroup. The goal of redesignating high-performing ELL students as language-proficient students causes high achievers among ELL students to exit the subgroup. The consequence is downward pressure on ELL test scores, worsened by the addition of new ELL students, who are typically low achievers. * Factors outside of a school's control. CRESST research shows substantial nonschool effects on student learning even within ELL subgroups. Schools are therefore unable to control all the factors related to student achievement. We elaborate on these ELL issues below and offer some suggestions to help schools meet the NCLB goals. Our comments are based on a series of research reports by Jamal Abedi and others. Low Performance and Slow Improvement CRESST research, supported by the National Assessment of Educational Progress (NAEP) and state test results, shows that English-language learners consistently perform lower than other students and frequently lower than many other subgroups. The Massachusetts Comprehensive Assessment System (MCAS) (a standards-based assessment chosen partly because it has collected six years of data) offers a good example. In 1998, the first year of MCAS, only 7% of ELL students in the Boston Public Schools and 8% of ELL students in Massachusetts overall reached the level of proficient or above in 10th-grade English language arts. This compares unfavorably with the statewide figure of 38% proficient or above for all students -- a gap of approximately 30 percentage points. In the next few years, perhaps spurred by the adoption of the 10th- grade English language arts assessment as a graduation requirement,1 Massachusetts 10th-grade English language arts scores improved substantially, reaching 61% proficient or above statewide in 2003. However, by 2003, the gap between ELL students and Massachusetts students overall had increased to 49 percentage points: 61% for all students versus 12% for ELL students. In Boston, the state's largest district, with approximately 10% ELL students, the gap grew as well, beginning at 11 percentage points in 1998 and increasing to 20 percentage points in 2003. Rapid progress by students overall, combined with policies that test ELL students who have lived in the United States for very short periods of time, both have contributed to a growing ELL achievement gap in many states and school districts. …

マサチューセッツ州におけるテスト政策と教育アセスメント行政の実態--「マサチューセッツ州総合評価システム」の成立と影響

マサチューセッツ州におけるテスト政策と教育アセスメント行政の実態--「マサチューセッツ州総合評価システム」の成立と影響

A View from the Bottom: What Happens When Your School System Ranks Last?

When the superintendent of the Lawrence Public Schools decreed that test scores must be raised, the district added two time-consuming test- preparation programs to the curriculum. Mr. Vogler and Mr. Kennedy recall the difficult choices that their school had to make in order to accommodate the additions to an already-crowded schedule. WITH FEAR, apprehension, and foreboding, we opened our morning papers. Were they in there? Sure enough, the test results were right on the front page. As we quickly scanned the results for the Massachusetts Comprehensive Assessment System (MCAS) for 2000, our fears were confirmed. Once again, our school system was ranked last in the state, the third year in a row. Today, one of the most controversial issues in public education is the use of standardized tests as the focal point of state accountability programs, specifically, the use of state-mandated, high-stakes tests. The most vocal opposition to such testing programs has focused not so much on the tests themselves but on the attached to the outcomes. These high stakes sometimes involve incentives, such as cash awards to schools or individual teachers whose students demonstrate high levels of performance. But they have also taken the form of the public reporting of test results, the prevention of grade-to-grade promotion and high school graduation, and the possible takeover of schools that continue to demonstrate low levels of student performance. Although the debate surrounding high-stakes testing programs continues, the reality for most public school administrators, teachers, parents, and students is that the tests are here, they do count, in some cases students must pass them to graduate, and in almost all cases the quality of a school system will be judged by how well students perform on the tests. In public school systems where students perform well, the results will be taken as proof that the schools are providing a high- quality education. But what happens to a public school system in which the students don't perform well on these tests? Are these schools not providing a high-quality education? What is a poor-performing school system to do? As a teacher and a former teacher in the Lawrence (Mass.) Public Schools, we have the dubious distinction of having been part of a school system that ranked either last or second to last in terms of student performance on the MCAS in every year since the test scores were reported to the public. But before we discuss the programs implemented to improve the students' MCAS scores, we'd like to offer some background on the community, the student population, and the Lawrence public school system. * Although there is a strong regional economy, Lawrence is the 23rd- poorest community in the country with a population of more than 50,000. Its unemployment rate in 2001 was 8.8%. * The most recent waves of immigrants to the city are from the Caribbean, Central and South America, and Southeast Asia. * Forty-three percent of adults in Lawrence did not finish high school. * Eighty percent of parents in Lawrence identify a language other than English as the primary language used in their homes. * The pre-K-12 student population of 12,562 is 86% minority (primarily Latino); the district has the highest percentage of Latino students in the state. * Seventy-seven percent of the students enrolled in the Lawrence Public Schools live below the federal poverty level. * About one-third of the students receive some form of federal assistance. * Almost one-third of the students drop out. * The Lawrence Public Schools have had five superintendents in the last eight years (two were interim). * Lawrence has the only nonaccredited public high school in Massachusetts. Our current superinten-dent, a former administrator in the New York City schools, is quite familiar with high-stakes testing programs (the Regents Examinations). …

Is it a House … Or a Pile of Bricks?

IN 1996 the Maine legislature established the Learning Results, comprehensive set of standards to be achieved by all students.1 The legislators further decreed that student achievement of these standards must be measured by combination of state and assessments. Toward this end, Maine school leaders now face the daunting charge to develop local systems. It is not yet clear whether the Maine Department of Education will subject district's to procedural review and approval, but the stakes are high nonetheless: by 2007, high school diploma in Maine will be tied to achievement of the Learning Results, and systems are to provide the means for certifying this achievement. To be sure, the welcome upside of these developments is that school leaders in Maine are being asked to meld state and achievement information for certifying achievement of state standards. This synthetic approach runs counter to the practice in 18 states -- soon to be 24 -- where high school graduation is linked inextricably to passing state test.2 However, clear downside of the Maine policy is the absence of clarification -- from the state legislature, from the experience of other states, or from the professional literature -- regarding just what is. While the term local is clear enough, assessment system is often used in reference to state's testing program. For example, the Massachusetts Comprehensive Assessment System, standards-based state test, is administered to all students in grades 4, 8, and 10.3 So again, just what is local In particular, what makes such in fact, system? This is the fundamental question with which Maine school leaders are currently wrestling. The Maine Department of Education formed an advisory group, of which I was member, to deliberate this question. Fueled by our resolve to provide helpful guidelines to school leaders and informed by the thoughts of others,4 we identified what we believed to be important features of system. Insofar as Maine is not alone in calling for the creation of such systems, our observations may provide guidance for school leaders in other states as well. Indeed, as more and more states begin to revisit the advisability of single-test graduation policies,5 the notion of an may be seen as an increasingly attractive alternative to high-stakes testing. This is now even more likely as consequence of the No Child Left Behind Act, which allows states to use combination of state and assessments in satisfying the requirement for annual testing in grades 3 through 8.6 What Is Local Assessment System? A system, the American Heritage Dictionary tells us, is a group of interacting, interrelated, or interdependent elements forming complex whole. Elements in must cohere. Thus is coherent, coordinated plan for assessment. It is constellation of measures that, together, yield data that document progress toward student mastery of announced learning targets. Clearly, the is made up of individual assessments. But collection of assessments does not entail any more than pile of bricks constitutes house. Therefore, the fundamental question for school leaders is this: In what sense does their plan constitute of assessments, rather than mere collection of assessments? In the view of the advisory group, has at least six important features. First, the assessments collectively are relevant to announced learning targets. That is, provides evidence of student achievement regarding formally specified, rather than tacit or implied, learning targets. Further, these learning targets are stated with sufficient specificity to communicate measurable outcomes -- outcomes that are directly amenable to measurement and assessment. …

The Human Face of the High-Stakes Testing Story

Schools like the Boston Arts Academy are desperately trying to keep the Massachusetts Comprehensive Assessment System from destroying their very fabric, Ms. Nathan says. She intends to make it through the MCAS mania by continuing to fight for a rich and rigorous arts and academic curriculum, taught by highly qualified and committed teachers in an atmosphere of respect and high expectations. THE NEWS stories, editorials, and academic debates about high-stakes tests deal mostly in ideologies, generalizations, and statistics. What's easily lost in the palaver is the reality of school for the people who have the most to gain or lose -- the students. In Massachusetts, the reality that faces students now includes the imminent threat that they will not receive a high school diploma unless they pass the 10th-grade exams in the Massachusetts Comprehensive Assessment System (MCAS). The numbers are grim, even with this year's much-heralded improvement in scores: close to half of the students in urban high schools are still failing the test; among black and Latino students, the failure rate is between 60% and 70%. But who are these students, really? do they know and not know? are their strengths and weaknesses, and what are their possible futures? I know some of these young people very well. They are students at my school, the Boston Arts Academy (BAA), a three-year-old public high school of visual and performing arts. They are products of the Boston Public Schools. They are typical, yet each one is unique. And their stories shed a different light on high-stakes testing. Let me introduce you to two of them. Tony's Story Tony first caught my attention by standing over me in the school cafeteria one day as I ate lunch with some students and teachers. He glared at me, put his hands on his hips, and stomped his foot a little. I turned to him. school isn't teaching real science, Ms. Nathan, he said. You need to do something about it. The other students looked up from their tater tots and turkey nuggets, glancing back and forth from me to Tony. The teachers stopped eating too. This was during the fall of the Boston Arts Academy's first year. We'd been open for all of two months, and no one had challenged me in quite this way before. I was acutely aware that my response would have long-lasting significance. What is it you object to, Tony? I asked, trying to stay calm. First of all, he said, there are no textbooks. The teachers give us these readings -- novels and stuff. is that? And I'm in a class with freshmen and sophomores. I've already done my freshman year. I don't get why I'm in a class with freshmen. Don't I know more science than them? Anyway, we're just not doing science. I mean, how is building an energy-efficient house science? And then there's this humanities stuff. is that? Where's English? Where's history? My friends at Central High aren't studying China like we are. And the math. Come on, Ms. Nathan. Where are the problems? It's all reading! I used to be an honor-roll student and get 90s or 100s on all the tests. I haven't even had a test here! Tony paused for breath, and I took a deep one, too. Wow, you have a lot of issues, I said, with a voice as steady as I could muster. I asked Tony to sit with me. Then I asked him a little about the classes at his previous high school. He described them pretty much the way I thought he would. Each week the teacher would assign a new chapter from the textbook, and the students would answer the questions at the end. The technique was the same for all subjects. Tony had never experienced what might be called active, hands-on, or open-ended learning. He had certainly never had to read primary-source documents as he was now doing in his humanities class (English and history) at the Arts Academy. He had probably never heard anything about ancient China before. …

Online Assessment: Moving beyond ‘Gotcha’

ONE OF the major criticisms of state assessments is the widespread perception that they're out to get someone, be it schools, teachers, principals, or students. Even as they attempt to hold high expectations and remain fair in comparing apples to apples and oranges to oranges, many state assessment systems have become increasingly rigid in structure and frequently trail plumes of unintended consequences. Teachers often don't receive test results soon enough to allow them to adapt their instruction to meet individual student needs, and so the tests tend to function as snares for weaker students. To lessen this gotcha factor, a number of states are working to correct these problems, and a few of them are beginning to improve their capacity to assess students via computers. South Dakota, Oregon, Massachusetts, Kentucky, and Virginia are taking different yet interesting approaches. South Dakota. South Dakota is out in front of the pack. A new law, S.B. 234, mandates that every district administer the same criterion- referenced test - once in the fall and again in the spring - to students in grades 3, 6, and 10. In response to this mandate, the South Dakota Department of Education and Cultural Affairs contracted with EdVISION to develop the Dakota Assessment of Content Standards (DACS). The DACS is a standards adaptive measurement assessment, tracking student mastery or nonmastery of learning objectives by determining the point of frustration in each instructional unit. The system adjusts the test items according to each student's ability level. There is no out- of-level testing, and each student receives a unique test. During the 2000-01 school year, districts piloted the DACS in reading and math. By now, the tests in science and language arts should be available, and, according to the new law, the DACS will be mandatory by this spring. Oregon. Three specific problems with the assessment system have emerged in Oregon: managing the test booklets, answer sheets, and other materials; returning scores so late in the school year that they are not useful for adapting instruction to meet student needs; and the difficulty in testing new students, retesting students, or providing a test at the time a particular student is ready to demonstrate proficiency. The state department set up a task force that commissioned a feasibility study; reviewed research on computer-adaptive testing; met with education organizations to gather input about the needs and capabilities of schools; met with personnel from 17 districts and three educational service districts, which represent the state's geographic areas and range of district sizes; and analyzed implementation issues. Based on the analysis and feedback, the task force designed a new system. The Technology Enhanced Student Assessment system will be computer- based, with access to a central server at the state department, and will work at multiple sites within a school on either Macintosh or Windows-based platforms with Internet connections. The first phase of the new assessment system has focused on reading and mathematics in the multiple-choice format, beginning with 20 high schools and 10 elementary schools. The goals are to eliminate test booklets and answer sheets, save time because fewer test questions can produce a more accurate score, return scores immediately, test new students at enrollment, offer retests, improve student motivation to perform well on the tests, challenge students appropriately with questions based on prior responses, enhance test security, store test records, and update them immediately. Massachusetts. The state department in Massachusetts is focusing on how to help students improve their skills in math and language arts so that they can succeed on the MCAS (Massachusetts Comprehensive Assessment System). In October 2001, the department contracted with the Princeton Review (a provider of test-preparation and college-admissions services) to make the Princeton Review's online testing and assessment tool, Homeroom. …

Learning from the Experience of others

AS THE YOUNGER of two children, I vividly remember day my brother was caught with and made to lie across our parents' bed to suffer consequences - three or four swats of the belt. I huddled outside door and added playing with matches to a mental list of behaviors I should avoid. I'm happy I took those mental notes. Today, as states consider behaviors they should and should not pursue to improve their education systems, it might be helpful if some of us sat outside door and took notes. After all, state after state has dedicated substantial thought, time, energy, and dollars to raising levels of student achievement, through passing individual bills and omnibus legislation that attempts to fix entire systems. The importance of omnibus bills should not be underestimated. Nor should these bills be confused with bills, which have acquired their name because lawmakers start hanging little extras onto some piece of legislation and so leave original bill buried in a mishmash of frequently unrelated pet provisions. Unlike Christmas tree bills, omnibus bills retain their focus on an issue such as education, but, instead of taking a piecemeal approach, they address whole system, including such items as funding, level of expectations for students, discipline, licensure, and governance. They launch systemwide changes. They are important. That is why, to help evaluate impact and implementation of such bills, a growing number of states have included provisions that require monitoring of these major reforms. These states are installing their own versions of sibling observers to huddle outside door and take notes on consequences of legislation, whether intended or unintended. One example is Massachusetts Education Reform Act (MERA), which was passed in 1993. It included a provision for a commission to oversee implementation of MERA, whose effects now seem to be hitting fan. This oversight commission, Massachusetts Education Reform Review Commission (MERRC), is supposed to provide objective, policy-relevant research and analysis to legislators and other key players in Massachusetts' education reform efforts. The commission's primary activity is funding independent research on various topics related to education reform. Its members include teachers, superintendents, principals, representatives of higher education (both private and public), parents, legislature, and school committees. The commission also aims to facilitate discussion between researchers and policy makers. In 1999 commission sponsored its first working conference of researchers and policy makers. The objective was to begin to develop strategies for stimulating additional research activity related to MERA. Last November, MERRC released a report titled How Massachusetts Schools Are Using MCAS [Massachusetts Comprehensive Assessment System] to Change Curriculum, Instruction, Assessment, and Resource Allocation, in which it reported results of a survey of school and district staff members and found following: * More than 90% reported that they receive results for students who are tested in their building; * 82% indicated that they were personally involved in analyzing their school's or district's MCAS results; and * more than 70% reported that their use of MCAS data affected instructional strategies they use in classroom. More recently, MERRC released its first annual report, and it is currently taking proposals from independent contractors to pursue several new aspects of research.1 For 2001 annual report, researchers visited 30 schools and interviewed superintendents, principals, teachers, and parents. Others, such as legislators, department of education personnel, and members of professional associations, were interviewed by phone. In all, researchers talked with more than 200 teachers, 90 superintendents, 100 principals, and 100 parents. …